<style> .smal { font-size: 70%; } .extra-smal { font-size: 30%; } .invis { color: rgba(0, 0, 0, 0.0); } .multiCol { display: table; width: 100%; text-align: left; } .col { display: table-cell; vertical-align: top; padding: 2% 0 2% 3%; } .col:first-of-type { padding-left: 0; } .reveal h3 { text-transform: none; } .reveal h1 { text-transform: none; } .reveal { font-family: "IBM Plex Sans" } </style> # Filecoin and Drand The quest for unbiasable randomness. </br> <div class="smal"> Kuba (<a href="https://github.com/Kubuxu">@Kubuxu</a>) https://hackmd.io/@Kubuxu/FilecoinDrand </div> --- ### Problem The Expected Consensus is a high throughput, low latency method to achieve consensus. <div class="invis"> The Expected Consensus requires randomness which cannot be biased by actors in the consensus. </div> --- ### Problem The Expected Consensus is a high throughput, low latency method to achieve consensus. <div class=""> The Expected Consensus requires randomness which cannot be biased by actors in the consensus. </div> --- ### Problem Initial design of Filecoin used integrated randomness source in form of Verifiable Random Function (VRF) chain. <div class="invis"> Actors playing the consensus game can make decisions affecting value used as randomness for the next epoch. </div> --- ### Problem Initial design of Filecoin used integrated randomness source in form of a chained Verifiable Random Functions (VRF). <div class=""> Actors playing the consensus game can make decisions affecting value used as randomness for the next epoch. </div> --- ### Solution Overhaul the security model, redesign consensus mechanism. <div class="1invis"> Invest time and energy to bring drand </div> --- ### Why Committed Capacity sectors represent 99% of Filecoin's 15EiB storage. <p class="invis">Currently available upgrade pathway is equivalent to sealing a new sector.</p> <p class="invis">Sealing a sector is high-latency process. We can reduce that delay with use of existing replica.</p> Note: the number of deals is going up every day and SnapDeals has the potential to accelerate it immensely --- ### Why Committed Capacity sectors represent 99% of Filecoin's 15EiB storage. <span class="">Currently available update pathway is equivalent to sealing a new sector.</span> <span class="invis">Sealing a sector is high-latency process. We can reduce that delay with use of existing replica.</span> Note: the primary drawback of current update system, requirement to seal a new sector --- ### Why Committed Capacity sectors represent 99% of Filecoin's 15EiB storage. <span class="">Currently available update pathway is equivalent to sealing a new sector.</span> <span class="">Sealing a sector is high-latency process. We can reduce that delay with use of existing replica.</span> Note: requires producing the replica, get client data in a sector, on chain, in way shorter timeframe Sealing a sector is high-latency due to the computation needed to verifiably commit capacity to the network -- which means that adding new storage deals to Filecoin is currently slow. --- ### SnapDeals SnapDeals are a lightweight way of updating a Committed Capacity sector with Clients' data. <span class="invis">Storage Providers can use an existing sector, instead of sealing a new replica.</span> <span class="invis">Avaliable after the OhSnap upgrade on March 1st.</span> Note: leverage existing properties of the CC sector --- ### SnapDeals SnapDeals are a lightweight way of updating a Committed Capacity sector with Clients' data. <span class="">Storage Providers can use an existing sector, instead of sealing a new replica.</span> <span class="invis">Avaliable after the OhSnap upgrade on March 1st.</span> --- ### SnapDeals SnapDeals are a lightweight way of updating a Committed Capacity sector with Clients' data. <span class="">Storage Providers can use an existing sector, instead of sealing a new replica.</span> <span class="">Avaliable after the OhSnap upgrade on March 1st.</span> --- ### Benefits <div class="multiCol"> <div class="col"> Time to deal activation reduced from 5-6 hours to **30 minutes** or less. <span class="invis">Computational overhead of sealing step is skipped. Simple one-message update protocol. </span> </div> <div class="col"> <img src="https://i.imgur.com/VnxACND.png"> </div> </div> Note: in comparison to current sector update or sealing ^^ this doesn't show up on the slide ^^ --- ### Benefits <div class="multiCol"> <div class="col"> Time to deal activation reduced from 5-6 hours to **30 minutes** or less. <span class="">Computational overhead of sealing step is skipped. Simple one-message update protocol. </span> </div> <div class="col"> <img src="https://i.imgur.com/VnxACND.png"> </div> </div> Note: one-message increases the reliability of the protocol --- ### Benefits Quicker onboarding benefits Clients and allows for new use-cases. Storage Providers can upgrade their existing sectors with FIL+ deals without resealing to 10x their QA power. --- ### Benefits Storage Providers gain the ability to reuse a sector's pledge collateral without resealing. Data onboarding can happen at higher throughput. Unlocking 12EiB of latent storage space on Filecoin. Note: This allows large storage clients to onboard their data onto Filecoin quicker. using CC as a buffer --- ### Future - Repeated Sector Updates - Reduced proving overhead - Even faster onboarding --- ### How Encode data into Committed Capacity sectors $R^{*}_i = R_i+D_i\times \rho_i \mod \mathbb{Fr}$ and prove the correct execution of the encoding step. <span class="smal"> [FIP-0019](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0019.md) </span> --- ### Security The replica is incompressible, the update process has to preserve this property. The $\rho_i$ algorithm is key. $\rho_i = \text{Poseidon-128}(CommD, \lfloor\frac{iH}{N}\rfloor)$ Note: ideal algorithm for \rho_i would be pure randomness, --- ### Reducing required messages The protocol, initially, required three messages for two rounds of interactive randomness: - Encoding randomness $\rho$ - Challenge generation The interactive randomness added 3 hours of delay to the protocol. Note: major blocker which reduced the usefulness --- ### Reducing required messages Grinding resistant encoding allows for non-interactive encoding randomness. Fiat-Shamir transformation allows for non-interactive challenge generation. SnapDeals is a non-interactive, one message protocol. <span class="smal"> [FIP-0019](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0019.md) </span> Note: we've overcame this issue by improving the underlying protocol --- ### Implementation Written in Rust: [`storage-proofs-update`](https://github.com/filecoin-project/rust-fil-proofs/tree/master/storage-proofs-update) Uses a suite of tools and libraries created by Filecoin: - Large data processing - Merkle tree building - CPU and GPU optimized hashing - Big integer and elliptic curve arithmetic - Cryptographic proof generation --- ### Proving SnapDeals Storage providers prove that they updated a CC sector correctly via **Groth16**. Small proofs attesting to the correctness of large computations. Computations must be expressed as a precise sequence of arithmetic operations called a **circuit**.  --- ### Trusted-Setup Large multi-party computation generates secure proving and verifying keys for each circuit. Each participant published an attestation: [`phase2-attestations`](https://github.com/filecoin-project/phase2-attestations/tree/master/934fe8c), which can be used to verify Filecoin's Groth16 keys.  --- ### SnapDeals Runtime | Sector Size | Replica Update | SnapDeals Proving | | :---------: | :--------------: | :---------------: | | 32GiB | 5m38s | 7m52s | | 64GiB | 9m19s | 8m17s | * Using NVIDIA GPUs, $\geq$ 256GB RAM, $\geq$ 24 cores --- ## Thank you!