# **Treasury Proposal: parasnaps.io maintenance**
**Beneficary: DDbQ9YnjeLS6DvZgvTgFvttnqJZrD12gVq4889yh9opXVS6**
**Date: 18.4.2023**
**Requested Amount: 860.22 KSM/27'041.12 USD (EMA7 31.435$)**

**Link: https://amforc.com/parasnaps**
**E-mail address:** [e-mail]
**Twitter:** [@amforcag](https://twitter.com/amforcag)

**Short description:**
To be able to recover inside an epoch on Kusama and prevent an offline event or potential slashing, we built a snapshot solution that allows bootstrapping a Kusama/Polkadot validator in less than five minutes to continue producing blocks.

---

# 1. Context of the proposal
Parasnaps was built out of the necessity to have a high-throughput snapshot solution to be able to take fast actions. Existing snapshots solution took around ~40 minutes to Download and our solution allowed to restore a validator in a couple of minutes.
### Team credentials
We are experienced in running validators since NPoS was enabled. Early 2021, we founded our own company which provides infrastructure solutions focusing on the Polkadot/Kusama ecosystem. The company is self-funded, independent, domiciled and based in [Switzerland](https://sz.chregister.ch/cr-portal/auszug/auszug.xhtml?uid=CHE-277.593.688).

### Community tips
Parasnaps.io was reported by community members for a [tip](https://www.dotreasury.com/ksm/tips/11806490_0xcbd0dd314675d31d537f456ca3e06104e4e2c23b6bb5dbe453f948f7d29c39a8) from the Kusama council. It was accepted and approved by the council.

### Treasury Proposal
Maintenance for the first six months of Parasnaps was covered by the Kusama [treasury](https://kusama.polkassembly.io/treasury/203).

### Central topic
Parasnaps was launched in February 2022, this proposal aims to cover the costs and work generated by Parasnaps from August 2022 until its decommissioning in March 2023.

# 2. Problem statement
For every session, validators must send an “I’m online” heartbeat if no blocks were produced to indicate that they are live during an epoch. It is possible that the heartbeat is not sent until 60-70% of the epoch is over. Slashing would occur if more than 10% of the network is unresponsive.

With an epoch period of one hour on Kusama, this means that there could be the scenario with less than 20 minutes to react to a potential issue and prevent a slashing event.

Other potential issues are bugs in either rocksdb or paritydb – because of this, both DB versions must be available at all times.

# 3. Proposal Objective/solution/s to point 2
The solution we built uploads daily synced snapshots for Polkadot and Kusama for both DB versions. Those snapshots are synced across five different geographical locations to prevent local outages and increase the download speed.

### DB sync/testing
Snapshots are always synced from scratch if the snapshot machines are updated to a new release or if there are any changes to the DB. Before making a snapshot available to the public, we run a validator with rocksdb and paritydb for at least 24h to check for any potential issues. This led us to the discovery of multiple issues:
- [paritydb brekas on upgrade](https://github.com/paritytech/polkadot/issues/5168)
- [StateDb does not have PRUNING_MODE](https://github.com/paritytech/polkadot/issues/5554)

### Download
There are three ways to download the snapshots as mentioned on the website with examples:
- Interactive (Bash script)
- Manual
- Ansible (examples on how to integrate into the automation)

All three options rely on a Rust binary that executes 16 concurrent downloads and keeps the results in a sorted binary tree to allow piping into stdout to directly be extracted.

### Demo
This is the same demo from the inital treasury proposal, which is rocksdb for Kusama with a size of 154 GB as per August 4, 2022.



The screenshot above shows the directory size before and after the download, as well as the time that it took to download the 154GB, which was 4 minutes and 50 seconds.

### Who does this solution help?
- Validator operators that need to failover in a fast manner.
- Validator operators that want tested snapshots.

### Usage
From merging the monthly usage reports for the s3 buckets (see below), a total of 43'332.8 GB was downloaded in the time period. Assuming an average of 170 GB for a download, this results in ca. 255 downloads in the time period, which averages to around ~1.1 downloads per day.

### Operational expenses
Parasnaps is hosted on AWS. We marked all related services with a corresponding tag to be able to create a report for the proposal. In October, we switched from VMs for the syncnodes to baremetal, due do the increasing resource requirements of the chain. In addition to the infrastructure cost, we included an hourly rate of $85 for labor.
For the operation and maintenance we estimate 4 hours a month, which would equal to 32 hours for the last eight months. This includes operation of the related services and machines, taking care of it and ensuring it runs without issues.
For testing of new snapshots and troubleshooting issues, we estimate a lower bound of 5 hours a month, which would equal 40 hours for the last eight months.

# 4. Payment conditions:
Specified payments below are a reimbursement for Parasnaps for the period from August 2022 to March 2023.

### Amount
750.43 KSM/27'041.12 USD (to be converted using 7d EMA on the day of the submission)

### Breakdown

| Item | Cost | Notes |
| -------- | -------- | -------- |
| Parasnaps Domain | $0 | Already paid by treasury |
| Infrastructure | $16443.41 | |
| Baremetal | $1348.8 | 2x Baremetal for October-March (104.- CHF per node a month) |
| 32h maintenance | $2720 | Estimated labor from August-March for maintenance (see [here](#Operational-expenses)) |
| 40h labor | $3400 | Estimated labor from August-March for troubleshooting and testing (see [here](#Operational-expenses)) |
| Total excl. VAT | $23'912.21 | |
| VAT 7.7% | $1841.24 | Swiss VAT (verify [here](https://www.estv.admin.ch/estv/en/home/value-added-tax/vat-rates-switzerland.html)) |
| Total incl. VAT | $25753.45 | |
| Exchange Rate KSM/USD 7d EMA | $31.435 | |
| Slippage + conversion fee 5% | $1287.67 | |
| Total + slippage | $27'041.12 | |
| Proposal KSM | 860.22 KSM | |

### How does this amount cover the work of the proposal?
The calculations above summarize the operational costs to offer this solution publicly for the community over the last eight months. With the proposal, we can cover the costs incurred over the last eight months.

# 5. Future Proposals:
With warp sync in its current state, we believe Parasnaps is no longer required and has been decommissioned in March 2023.