SWAP pricing - HackMD

# SWAP pricing # SWIP --- SWIP: TODO: update title: TODO author: Daniel A. Nagy <daniel@ethswarm.org> (@nagydani) discussions-to: https://swarmresear.ch/t/postage-ex-proof-of-burn/ status: Draft type: Standards Track category: Core created: 2019-07-12 ---  # SWAP pricing ## Simple Summary  We describe a protocol which nodes use to communicate their price for delivering chunks in the Swarm network. On top of this protocol, strategies can be implemented by nodes who wish to compete with other nodes on quality/price. A minimal strategy is presented which makes nodes request different prices, depending on the proximity of the requested chunk. While this strategy implies that nodes don't initiate price changes by themselves, they need to react to price changes from upstream peers when it is essential or profitable to do so. The price signalling is closely coupled with actual retrieve requests and therefore is formulated as part of the retrieve protocol ## Abstract  ## Motivation  This SWIP lays the foundation for a Swarm network where the price for the Swarm accounting Protocol (SWAP) is dynamic and based on local decisions. This is essential for the following reasons: - Bandwidth costs are not heterogenous around the world. Allowing nodes to express their cost-structure via their price will enable competition on price/quality, ultimately benefitting the end-user of Swarm. - Bandwidth costs are constantly changing. Being able to react directly to these changes is essential for a node as without this possibility, the node operator might decide to shut down their node when costs go up or end-users might overpay for an extended period of time when prices decrease and there is no competitive pressure for nodes to reduce their price as well. The choice was made to leave out the implementation of an active response to changes in cost structure / bandwidth usage, as there exists an economic incentive for nodes to use a strategy that is better than we ever think of. To facilitate such tinkering, a minimal and passive reaction to price changes by other nodes is included. Ultimately, we expect competetive pressure to automatically fade out our presented passive strategy. ## Specification  This specification distinguishes between protocol and strategy. The protocol *must* be implemented and cannot be different between peers in the same network. A strategy should be implemented, but because different strategies can co-exist on the same network, it suffices to start with the most minimal strategy. ### Protocol TODO: describe the specific encoding/serialization of the messages. - `RetrieveRequest: {price, chunkAddress, Ruid}`. Sent by a requesting peer to an another peer to request a chunk. `price` is defined by the strategy and should be the best-effort guess by the originating peer about the price for deliverying the chunk. `chunkAddress` is the Swarm hash of the desired `chunkContent`. A `RetrieveRequest` may be answered by a `ChunkDelivery` or `NewPrice` message (described below). Upon receiving a new price message, requestor can rerequest, now with the given new price. Measures must be taken not to retry endlessly after receiving a `NewPrice` message. - `ChunkDelivery: {chunkContent, Ruid}`. Sent by a node that has got the chunk to deliver a requested chunk. The applied price by SWAP is the `price` from the `RetrieveRequest` with the same `Ruid`. - `NewPrice: {price, chunkReference}`. Sent by an upstream peer to a downstream peer to signal that for this particular `chunkReference`, the downstream peer has to pay `price`. The `chunkReference` may be synthetic, meaning that there is no actual `chunkContent` mapping to the `chunkReference`. A peer that receives a `NewPrice` message is supposed to update it's state (see below). ## Strategy The minimum strategy to be implemented is the strategy which sufficiently incentivizes other people to come up with better strategies, but which not The following strategy is a basic way on how nodes *can* use the protocol described above. The strategy intends to be as easy as possible to implement, while balbalbala ### State Nodes keep track for each (historical) peer which prices they paid for each distance and when this price was last updated. Schematically, this mapping may looks like: ``` priceInformationRegistry: { peerID1 => priceInformation, ... , peerIDN => priceInformation } ``` where ``` priceInformation: [ proximity0: {price, lastUpdated}, ... , proximityN: {price, lastUpdated} ] ``` where `proximity` is `proximity` as defined in the [architecture docs](https://swarm-guide.readthedocs.io/en/latest/architecture.html#logarithmic-distance) between the peer and the `chunkReference` and `N` is the `saturation depth - 1`. ` ### Filling the state Nodes learn about the prices of their peers by trial and error: initially, the `priceInformationRegistry` is empty which means that the best-effort guess for the price of a `ChunkDelivery` equals zero. If storer nodes and/or forwarding nodes have a non-zero margin, the peer to whom the `ChunkDeliveryRequest` was send will answer the request with a `NewPrice` message, thus populating the `priceInformationRegistry` of the node. NOTE: Why not just putting it in the handshake? As the expected amount of hops is a function of the network size, it might happen that the price for a certain distance changes without ### Margin Margin is a configuration variable and it specifies the SWAP profit a node wishes to make for every `ChunkDelivery`. Hence, the price of a `ChunkDelivery` can be defined as, at least, the `margin` + the price of the cheapest upstream peer. ### Reacting to upstream price differences Nodes define their preferred supplier for chunks based on the following: For any chunk request: - Select the bin to forward the request to - Select the peer with the lowest price in that bin - Do a round-robin load balancing if there are multiple peers with the same price Based on this, we conclude that the reaction to a decrease in price for a peer in a certain bin is that we are more likely to send requests to him and that the opposite holds for an increase in price. ``` Note that a peer who has a very low margin and is connected to very cheap upstream peers might still be more expensive for a certain request than a peer with a high margin and expensive upstream peers, as the distance of the expensive peer to the chunk might be much less than the distance of a cheap peer to a chunk. ``` The other reaction is that when our upstream peers increase their prices, we need to increase our prices as well to still be able to get our desired margin. If the upstream peers decrease their price, we might want to decrease our price as well, as this way, we can get more traffic. ### Notifying about price changes It is in the interest of the network, that when there is a price change this gets propagated as soon as possible to the relevant parties. A price change can be initiated by a change of `margin` (by any upstream peer), or by a change of the network size. In both cases, a peer might decide not to notify other peers about the decrease in price--the only thing which changes is that it will accept `RetrieveRequests` with a lower price than before. Without notifying the downstream peers, however, a node might not get the expected amount of additional clients directly, as the downstream peers might never propose a lower price than before. For this reason, we propose that nodes pro-actively send `NewPrice` messages to his peers, essentially requesting them to update their `priceInformationRegistry`. In the case of pro-active `NewPrice` messages, the `chunkReference` may be synthetic, meaning it does not have to respond to an actual chunk; all that is important is that peers update their `priceInformationRegistry`. ### Overpriced bins Since the price of a node depends on the price of his upstream peers, it might happen that by change, all his upstream peer-connections in a certain bin are populated by expensive peers—effectively making the node itself to be too expensive as well for his downstream peers. To solve this issue, we propose to do statistical analysis: if a node has sensible prices for all distances, it is expected that half of his requests are in bin zero, a quarter in bin 1, 1/8th in bin 2, etc... up to (but excluding) the bin of the saturation depth. If a node receives statistically significantly less requests for one bin compared to the expected amount of requests in that bin, he marks the supposedly overpriced peers as non-functional and the hive protocol should kick in to suggest new peer connections. ### Reacting to demand increases/decreases A node that decreases its price such that he is the cheapest, is expected to get much more traffic. Without an adequate respose to an increase in traffic, a node might be forced to go offline as he is incapable of handling the increased traffic. In order to prevent this from happening, nodes must adapt their price actively, based on their bandwidth utilization. ### Swarmholes A Swarmhole is created by running one node (one database) with multiple peer identities. If a node has a sufficient number of peer identities, the expected amount of hops for any request to this node can be decreased to 0 (that is: he stores all data in the Swarm network). Due to competitive pressure, it is expected that nodes will create multiple peer identities (probably not to store the whole network if the network is sufficiently big) or collaborate in a centralized overlay network to share and pool requests (effectively achieving the same as one very big node). Such Swarmholes will decrease the price of `RetrieveRequest`s and decrease the latency. There are also problems: - Anybody who does not run a Swarmhole node (or is connected to a Swarmhole) cartel, will not be able to compete with the Swarmhole and hence cannot make a profit on SWAP. - Swarmhole nodesdesdes will be able to figure out which chunks belong together - Swarmholes can censor content or blacklist peers, making resp. the content more expensive and increase the price for the blacklisted peers (it/they must now route through "normal Swarm"). ## Rationale  TODO ## Copyright Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/). ## Questions: What is the price to serve a cached chunk? => minimum price == margin, but peers might offer a higher price because they don't know that you cached it. For some popular chunks, the price might be different than for non-popular chunk. What if the distance of the popular chunk is very far. How do I distinguish between NewPrice for the popular chunk and NewPrice for the non-popular chunk?

Syntax	Example	Reference
# Header	Header	基本排版
- Unordered List	Unordered List
1. Ordered List	Ordered List
- [ ] Todo List	Todo List
> Blockquote	Blockquote
Bold font	Bold font
Italics font	Italics font
~~Strikethrough~~	~~Strikethrough~~
19^th^	19^th
H~2~O	H₂O
++Inserted text++	Inserted text
==Marked text==	Marked text
[link text](https:// "title")	Link
![image alt](https:// "title")	Image
`Code`	`Code`	在筆記中貼入程式碼
```javascript var i = 0; ```	`var i = 0;`
:smile:		Emoji list
{%youtube youtube_id %}	Externals
$L^aT_eX$	L^aT_eX
:::info This is a alert area. :::	This is a alert area.