Last update: 7th Dec 2022
📌 Protocol Overview: We designed the protocol in this way:
In the following section we describe this idea in full details.
Clients: A client is the user of our protocol, it has data stored on a storage network (ie, Filecoin or IPFS) and wants to get an assurance on the fact that retrieval requests will be successful. To do so, the client can propose a retrievability deal to the providers in the protocol. At the high level, with this kind of deal the client pays a provider to lock down a collateral that be "slashed" (ie, taken from the providers) if the provider misbehave (more details on the deal syntax in the next section).
Providers: In our protocol, a provider is the party in charge of the retrieval service (ie, the client asks for a file, the provider serves it). When a provider accepts a retrievability deal, it locks down a collateral as guarantee of the retrieval service. This means that if a retrieval request is not fulfilled later on, the collateral will be taken from the provider ("the providers is slashed"). In the current implementation the list of providers participating in the protocol is part of the smart contract. In particular, this list is created and updated by the contract’s owners and a party that wants to act as provider in our protocol needs to communicate its intention to join to the owners. This can be done by filling the sign-up form available in the dapp.
min_price
: minimum price per second per byte,max_rate
: maximum rate (collateral/payment)max_size
: maximum file size accepted in a proposalmax_duration
min_price
= 0, max_rate
= slashing_multiplier
(here slashing_multiplier
is a protocol parameter, see "protocol parameters" section), and max_size
= 20MB.Referees: The referees are the core of our protocol, their role is to manage appeals from clients, to request file from providers and serve to clients, and eventually to agree on slashing a provider. The set of the referees has size (protocol parameter) and is chosen once for each instance of the protocol. In the current implementation, the list of referees is a part of the smart contract.
Smart Contract and Blockchain: the smart contract contains the logic to create the deals, create and process appeals and slash providers.
createDealProposal(string data_uri) -> uint256 deal_id
A client creates a deal proposal specifying:
data_uri
;payment
: amount (in native tokens) paid by the client to the provider if the deal is not invalidated by an appeal (ie, deal ends with no slashing); at the moment of the proposal creation, the payment
is "locked down" (ie, taken from the client's account and deposited to the smart contract); payment > 0
;duration
: for how long the deal is active, starting from when it is accepted (expressed in seconds); the proposal is valid only if duration
is in a range specified by the smart contract code (see "protocol parameters" section).collateral
: amount (in native tokens) required to be locked down from the provider account to accept the deal.Comment: In the current Client UI we designed two modes:
min_price * file_size * duration
. Then, the client can increase the two values as long as the following conditions are satisfied:
payment
≤ collateral
collateral
≤ max_rate
* payment
.A client can cancel a proposal (not accepted yet) at any time, and this action will release the payment
Moreover, a deal proposal has a timeout (currently set to 86400 seconds) after which it can not be accepted by providers anymore.
acceptDealProposal(uint256 deal_id, string data_uri)
A provider can accept a deal proposal if all the following is true:
collateral
.When the proposal is accepted, the collateral
is "locked down" (ie, from the provider's account deposited to the smart contract). Also, we timestamp this on-chain, and we consider the deal active from this moment (timestamp_start
).
Comment: we designed the provider CLI in such a way then before automatically accepting a proposal, the followinf conditions are checked:
file_size
and file_size
<= max_size
;min_price * file_size * duration
;max_rate * payment
.createDeal( address owner, string data_uri, uint256 duration, array[] appeal_addresses ) -> uint256 deal_id
A provider can create a deal by specifying:
data_uri
;owner
to this function so provider can assign deal to a clientredeemDeal(uint256 deal_index)
After a deal ends, if the provider was not slashed, then he can ask to get the payment
(ie from the smart contract vault, deposited to provider's account).
While the deal is active, the client can appeal to the referees at any instant on time for requesting the assured storage. To do so, one of the appeal addresses specified in the deal creates an appeal and the appeal is considered valid if all the following is true:
timestamp_start
and deal_duration
+ timestamp_start
);max_appeals
in the "protocol parameters" section);appeal_fee
. This is the amount of token that the client pays to the referees for their service. Currently
appeal_fee = committee_multiplier
* payment
The moment the appeal is created on chain defines the request_timestamp
.
If an appeal is created, any referee that is free can start the appeal process.
The moment the process is started on chain defines the origin_timestamp
.
The protocol to process an appeal is made by rounds each one of duration round_duration
(currently set to 300 seconds). In each rounds the referees try to retrieve the file in this way:
(deal_id + appeal_id + round_number)
;leader_waiting
time from when the round started, sends a “failure msg” on chain; the other referees see the message and do nothing for the remaining time of this round; in the next round all starts from Step 1;
(deal_id + appeal_id + round_number + 0)
;leader_waiting = 1/2 * round_duration
;After the rounds are over, the contract checks the total number of “failure messages”.
slashes_threshold
, then the provider is "slashed" (ie, the collateral
is deposited to the contract owner's account), the payment
is returned to the client and the deal is deactivated. Currently, we have slashes_threshold
= .DealProposalCreated(uint256 index, address[] providers, string data_uri)
: Event emitted when new deal proposal is created by the client;DealProposalAccepted(uint256 index)
: Event emitted when a deal proposal is accepted by a provider (defines timestamp_start
);DealProposalCanceled(uint256 index)
: Event emitted when a deal proposal is canceled by the client before being accepted;AppealCreated(uint256 index, address provider, string data_uri)
: Event emitted when new appeal is created by the client (defines request_timestamp
);AppealStarted(uint256 index, address provider, string data_uri)
: Event emitted when an appeal process is started by a referee (defines origin_timestamp
);RoundSlashed(uint256 index)
: Event emitted when a failure message is recorded during the process of an appeal;DealInvalidated(uint256 index)
: Event emitted when a deal is invalidated (ie the appeal process terminates with failure messages in each rounds);DealRedeemed(uint256 index)
: Event emitted when a deal is redeemed by the provider (needed to get the payment
).appeal_fee
= x payment
is paid to the referees each time that an appeal is made; current value: 0.2.
committee_divider = 5
and appeal_fee = payment / committee_divider
.slashing_multiplier
): the amount (in native tokens) slashing_multiplier x payment
is the maximum of the collateral that a client can ask for; current value: 1000.max_appeals
: maximum number of appeals per deal; current value is set to 5;rounds_limit
): number of rounds in the trial protocol; current value is set to 12.proposal_timeout
: timeout to accept a deal proposal (1 week), set to 86400 seconds (1 week).min_duration
: minimum duration of a deal (in seconds), set to 3600 seconds (1 h);max_duration
: maximum duration of a deal (in seconds), set to 43200 seconds (12 h);round_duration
: duration of a round of the appeal trial in seconds;slashes_threshold
: minimum number of "failure messages" required to slash a provider;Assumptions:
We need to show that a even when the client acts maliciously and colludes with referees, an honest provider (who sends the file to referees) is not slashed (with overwhelming probability).
Note that if there is at least 1 round where the leader is among the honest referees, then the retrieval is successful and the there is no "failure msg" on chain (ie, indeed if h > n/2, then and there is no enough dishonest referees to sig the failure massage in step 2). The probability of "at least 1 round where the leader honest" is:
With , we have . If , then .
Comment: a malicious user sending the appeal request makes the honest provider works twice. However this costs the appeal_fee
to the user, making this strategy irrational.
We need to show that a provider that does not send the file during the appeal trial will get slashed even if referees are colluding with him.
At each round, we have two cases:
slashes_threshold
messages and the provider is slashed.This section is ready to read :)
An investigation was done for informing the rational ranges of the protocol parameters as well as the soundness of the protocol dynamics as informed by an Bayesian representation and simulations based on the Deal Journey Process Diagram (fig 2). The main result was the definition of recommended ranges and values for the slashing_multiplier
, committee_multiplier
and max_appeals
protocol parameters, which are as follows:
committee_multiplier
is rationally upper bounded by the client belief on the retrieval utility. In practice, this boundary can be indeterminally large, and therefore an best gauge for decisions should be the UX, as fostering demand for the protocol is an consideration. We recommend therefore an value of with an acceptable region being between 0.1 and 1.0slashing_multiplier
is upper bounded by the provider belief on his change og being slashed and there's no trivial lower bound. The rational upper boundary is estimated to be between 1000 and 5000. Given that this parameter is a ceiling on the retrievability proposal, we suggest setting it close to the boundary. We recommend therefore an value of , with an acceptable range being between 100 and 2000max_appeals
will affect both client and provider expectations. Specifically, it can increase the probability of the provider being slashed and the client for getting back his payment. In that sense, max_appeals
is an slack parameter that allow us to compense the fact that the committe_multiplier
is way lower than what would be rational. We recommend setting-up to between 5, with an acceptable range being between 3 and 10Fig 2. Process Diagram for the Assured Deal Journey on the Retrievability Consortium
The process on which those conclusions were generated involved an variety of theoretical, computational and SME considerations. The point of departure is through formalizing Fig 2. through an set of equations describing each actor payoffs when engaging on their actions. Namely:
On the above formalism, we have the following terminology:
The following parametric assumptions were made:
Also, when building the formalism, there are the following modelling assumptions:
From the above pay-offs, an series of derivations can be retrieved, mainly:
In pratice, this means that deciding is dependent on the incidence of honest providers getting slashed AND it's effect on the marginal retrievability. Given SME priors and an Bayesian simulation, we believe that the equilibrium is between 1000 and 5000.
As for , it is coupled to the expected number of appeals. By introducing an ceiling on the max numbers, we can cut the long tail of the distribution and make the calculation easier. However it does have also an dependency on the leveraging, which we expect to be on the order of . With all of those together, we would expect to have an which can be quite large (on the order of 10). However, when taking the marginal retrieval probability and prior slash probability into consideration, the calculation becomes highly uncertain and with little actionable insight.
Therefore, we decide to use an UX consideration for deciding . Any value above 1 is going to be weird for the clients, as it means that they need to pay more to appeal than to have the assurance itself. This is also a point of departure from using agent rationality for parametric choice, as we now must make use of an global utility consideration: it is an design desirable that usage of the protocol is maximized.
The risk associated with low is about appeal spamming, which will interefere with the provider calculations. This is mitigated by having an max amount of appeals which one can do. Therefore, we recommend setting it up with an low value, like max_appeals=5
.
In order to make explict the argument on , we argue that we're also interested in maximizing , which represents the design objectives of the protocol. They are: