Farcaster and Bitcoin are very different networks but share a parochial approach to problem-solving. Bitcoin lets users store and move money without a trusted third party, while Farcaster does the same for text. Describing Farcaster as a decentralized, public, and text-centric social network would be accurate. Many apps have a "twitter-like" feel, with a text feed generated from a set of "follow" relationships. Developers are starting to push the limits of the current design, with apps focussed on image and video sharing, physical presence, and on-chain ownership. People want Farcaster to act more like Ethereum and less like Bitcoin, allowing more degrees of freedom and creativity. Could Farcaster become a "Turing-complete" protocol allowing different social networks to be built? Or is a unique architecture needed for each? Achieving Completeness Farcaster already has essential building blocks like decentralized identity, text storage, and graph relationships. But building diverse networks like Facebook, LinkedIn, YouTube, or Whatsapp requires a few more elements: Flexible schemas so that devs can introduce new content and metadata types without a protocol upgrade (e.g., polls) Context separation so that users can keep identities, graphs, and data segregated across networks. Long-term storage so that users don't have to discard older data which is valuable on some networks.

Problem Using a new app may may require an on-chain transaction. For example, signing up for Farcaster requires making a transaction to the Registry to get an fid. Using your primary wallet to make this transaction may be impractical for a few reasons: Your wallet is on your ledger, at home Your wallet is on your desktop, and you’re on your phone Your wallet is available, but you’re having trouble connecting You don’t trust this app, and would prefer not to have it come close to your wallet Solution The app creates a new wallet, uses a paymaster to cover the transaction costs and can charge you in fiat if necessary. The problem is that you now have a wallet for each new app and remembering the recovery phrases is impractical.

With the launch of Hubs now in its final phases, we wanted to recalibrate our priorities as a community. This document is a draft of some goals, problems and a few early solutions and we’re looking for input from casters. Goals Increase the number of people creating useful content on Farcaster. Increase the number of developers building on Farcaster. Achieve credible neutrality. Make money to keep funding protocol development. Stack-Ranked Problems Make sure that Hubs are working as expected

Users sign up for Farcaster by registering an fid using an Ethereum wallet. Apps can be authorized to act on the user's behalf by providing them with a signature from this wallet. Apps & Wallets The default line of thinking is that users should use general-purpose wallets to register, and apps can get signatures via WalletConnect. This is convenient for users who have a wallet and for app devs who don't want to deal with the headache of building a wallet. But in practice, the user experience is bad enough to hinder Farcaster’s adoption. In particular: Users without a wallet must set one up first before using Farcaster. Signing messages is confusing and intimidating to the average user. Wallet Connect does not always reliably establish connections.

Farcaster's identity layer comprises two contracts: a fully decentralized Id Registry and a moderated Name Registry. These contracts are on Goerli but must be deployed to a secure and stable network soon. Which one of the many L1 and L2's is the right home for our contracts? Requirements Farcaster Id Registry issues fids which are unique, numeric identifiers like 12398. It is important that: Registration is very affordable (< $1 in tx fees). Ownership is fully decentralized and cannot be revoked. Ownership is secure against motivated attackers.

Farcaster currently specifies the allowed lengths of strings in bytes. For example, a cast's text field can hold 320 bytes. Usually, this means 320 characters but because of how UTF-8 works, it can be lower. Currency symbols, fancy quotations and other common glyphs take up multiple bytes. Character lengths would be simpler to understand for most users. But they have other implications that affect protocol stability, developer experience and user experience negatively. Applications like Twitter do not use true character lengths, expose some of this complexity to users. What should Farcaster do? Proposal: Use Bytes Calculate cast text length in bytes and limit it to 320. The text hi would take 2 bytes, ħi would take 3 bytes and ħī would take four bytes due to UTF-8 encoding. Denote cast mentions with byte positions in a separate array in the object. Mentions do not count against the cast length and up to 10 mentions are allowed. The text hi @foo anđ @bar becomes {text: "hi anđ ", mentions: [{fid: 1, pos: 3}, {fid: 2, pos: 8}]} assuming foo and bar have fid's 1 and 2 respectively.

Thanks to @pfh, @adityapk, @sds and @kc for conversations that contributed to this design Farcaster Hubs promise to store ~20,000 messages for every fid perpetually. Registering an fid has a one-time gas cost, but hubs must pay ongoing cloud storage and bandwidth costs. A tragedy of the commons ensures since users have no incentive to minimize data transfer or clean up unused data, increasing the burden for hub operators. Our goal is to allocate limited storage and bandwidth between users to maximize network value. Our constraints are that Hubs must fit into a single cloud instance (< 64 TB) and we must support at least 10M users. We also believe that: Increasing costs may align incentives. While a one-time gas fee prevents rampant abuse, a recurring fee would result in useless data being flushed more often. Storage demand could exceed supply. Dynamic pricing or first-come first serve allocation will be needed to enforce the 64 TB limit. Premature optimizaton should be avoided. Charging too much, adding complexity or delaying launch may make the solution worse than the problem. Network costs are also a problem. Network throughput also places an economic burden on hubs, and a separate design is needed to handle that.

Inspired by repo made by kcchu: https://github.com/kcchu/buffer-benchmarks Benchmark script from pfh: https://github.com/pfletcherhill/farcaster-hub/blob/serialization-benchmarks/apps/benchmark/src/index.ts Step 1: Generate test data Generate 100 objects that represent deserialized CastAdd messages. Each has a 200 char text, two embeded URLs, two fid mentions, and a parent cast. Step 2: Encode test data for 15s For 15s, randomly pick one of the test objects and serialize it using the given framework (flatbuffers or protobufs). Record the number of messages encoded and the total size (in bytes) of the encoded messages.

Goals Casts should be “turing-complete” primitives that have enough degrees of freedom to allow developers to create new types of social applications. Casts should not fragment liquidity, and casts created in one application should render in other applications in a useful manner without needing constant work. These two goals and important to consider carefully because they often conflict with each other. For example, a totally open ended data schema would solve for (1) but violate (2). Current Solution Public social networks have posts which contains text, media or both. Media in this case is usually an image, video, URL or an audio clip in rarer cases. Farcaster has casts which have ≤ 320 characters of text and 2 optional urls. Note that our client today only supports one, but the protocol supports two and this will be fixed

Problem Users create signed messages and store them on eventually consistent servers called hubs. Messages must be referenced with a unique identifier in three contexts - from another message (message id), in the database (storage id) and in the sync trie(sync id). Our proposal recommended message id's like fid:timestamp:hash using a 32-bit timestamp and a very short 32-bit hash. Storage id's stretch the message id to avoid collisions with other data in the db and look like prefix:fid:postfix:timestamp:hash. Sync ids shuffle the message into timestamp:hash:fid to build a time-ordered merkle trie. Unfortunately, this leaves an attack vector open where a malicious actor could generate a collision. They could produce two different messages A and B that have identical ids. A hub that saw message A would never accept message B and vice versa. A client querying these hubs might get different responses for the same query. Modelling the Attack Vector A Farcaster message is a ~ 1KB binary object that is hashed with Blake3 to produce a unique identifier. If two different messages produce the same identifier (hash digest), it creates a problem for the network's sync algorithms.

Contributors: Paul Fletcher-Hill, Shane da Silva, Varun Srinivasan Why? Helps developers align on how Farcaster Messages should work. What? Specifies how farcaster messages should be constructed (the schema), how big they can become(sizes) and other restrictions (validation rules). Not: the rules of sets or x-message validation Message Represent any action taken by a user like making a post, etc etc Tamper-proof self authenticating objects signed by an fid contain a flexible data body that is defined by the sub type

Contributors: Paul Fletcher-Hill, Varun Srinivasan, Shane da Silva, Dan Romero Problem Farcaster messages should each have a unique identifier. Applications need unique id's to compare two messages quickly. Some messages, like replies and reactions need them to point to their parent. Databases can use them as a unique key to store and look up values. Today, Farcaster calculates and stores a 256-bit Blake2B hash on every message. When a reply wants to link to a message or a hub wants to store a message, they construct an identifier of the form <fid>:<hash> to uniquely reference the message. The fid prefix is not necessary for uniqueness but makes it easier to look up data sharded across many systems. Unfortunately, these identifiers are 512-bits long and take up a lot of space. Each additional byte on an id increases total storage costs by 1% since ids are repeated many times for indices. These ids also have no causal ordering, and Hubs resort to generating keys of the form <fid>!<timestamp>!<hash> to perform efficient time-sorting. Prior Art

Problem Farcaster Hubs are designed to store a copy of all the data on the network for all users. This will require an increasing amount of disk space, which raises the cost of running a Hub. If Hubs’ costs become excessive, we’ll see fewer Hubs which will lead to: The SMTP problem where new entrants are blocked out Collusion between hub operators to de-prioritize or censor content Attacks against hub operators to take down the network or force censorship These problem are particularly painful when the network is controlled by < 10 actors. It becomes very easy for collusion and/or targeted attacks. If the network approaches ~ 100 unique, geographically distributed actors, most of these vectors become impractical. We can state that the network is practically decentralized if we have ~ 100 unique Hubs operators. Hub Costs

In at least three situations as part of the Farcaster network, we are reconciling state on two distributed systems—Farcaster and Ethereum. These situations are: Farcaster SignerAdd and SignerRemove messages <--> ID Registry on-chain state Farcaster UserDataAdd messages <--> Name Registry on-chain state VerificationAdd messages <--> verification claims signed by Ethereum addresses In all of these situations, we defer to Ethereum state (i.e. SignerAdd messages are only valid if they are signed by the custody address in the ID Registry contract), which means that any consistency guarantees we may be able to make about the Farcaster network are dependent on Ethereum state. For example, given a set of Farcaster messages, two Farcaster hubs will merge the messages and come to the same state if they have processed the same events from the ID and Name Registry on-chain contracts. If they have processed different events, they may converge, but it is not guaranteed. Example 1: ID Registry state The ID Registry stores the current custody address for each fid. Users can add SignerAdd messages that authorize EdDSA key pairs to sign messages on behalf of the fid. However, SignerAdd messages for an fid are only valid if they are signed by the current custody address for the given fid.

Farcaster is a distributed network made of operation-based CRDTs. Unlike other CRDTs, the hosts of the network are potentially adversarial, which introduces some challenges. One challenge is that timestamps are not necessarily true representations of when operations happened—how do we differentiate between an old operation that is only now reaching our hub vs a new operation with a fake timestamp? One design decision we've had to make due to the unreliability of timestamps is signer revocation. When a signer is removed, we delete all operations ever signed by that key pair, rather than just preventing new operations. Example A user, Alice, has authorized two signers for her account, A and B. She's posted casts from both signers. Let's assume signer B is compromised, and Alice wants to revoke access for that signer. But she wants to keep the first two messages from that signer and revoke only the most recent one. This is her desired behavior: The problem with this approach is that Signer B can still create messages with timestamps prior to 5 and merge them because timestamps can be manipulated. The final message could be re-created by Signer B with timestamp 1 and would be considered valid.

Thanks to Dan Romero, Paul Fletcher-Hill, Cassie Heart, Aditya Kulkarni, Goksu Toprak and Shane da Silva for contributing to this proposal. A follow is the atomic unit of a social graph on networks like Twitter and Instagram. It is a literal request to add a user’s posts into your feed and an implicit endorsement that a user is worth listening to. Follows are a status symbol because the number and caliber of followers you have is a transitive proof of credibility. If many people choose to see your posts in their feeds, you must be an interesting person. The portability of the social graph is necessary for a decentralized network since it is an essential input into features like feed construction and recommendation. If moving clients required rebuilding your social graph from scratch, it would make it less likely that people would switch clients. Problem Outcomes follow incentives on decentralized networks, and the incentives to publish the follow graph correctly are weak. Applications may not publish follows, because publishing them is more work and makes it easier for users to leave. Users themselves may seek out such apps, because they may want the ability to see some users' updates privately. The problems arise when users want to switch apps and realize that their follow graph isn't portable. At this point, it is too late to resolve the issue and users may just stick with the old applications. Apps that behave incorrectly are rewarded by having better retention, which makes it harder for new apps to compete with them.

Users who sign up for Farcaster today go through a flow like this: The Merkle Farcaster app creates a new EOA for the user The EOA is used to register an fid and an fname The EOA signs a message which authorizes an EdDSA key pair as a signer A few people have suggested using smart contract wallets with EIP-1271. The assumption is that users already have an EOA (Metamask, Rainbow etc) which can be re-used. The flow would look like this instead: User deploys a contract from their EOA wallet As part of the deploy, the contract registers an fid and an fname

In an effort to achieve feature parity with the Merkle backend, we want to maintain a one-to-many mapping between verified ETH addresses and fids. An ETH address can be connected to a single fid, though an fid can have many connected ETH addresses. In order to do so, we will have to resolve conflicts in a deterministic way between VerificationAddEthAddress messages that reference the same ETH address. We already handle these conflicts between messages from the same fid, but the problem is more complex across multiple fids. Example problem One idea might be to continue to resolve conflicts solely using a LWW mechanic based on messages' timestamps. However, this method is bad, because two fids that both want to connect the same ETH address could continue signing messages after each other to flip the ETH address back and forth between them. In this situation, both fids would have gotten a signature from the ETH address at one point, and they're re-using that ETH signature with each subsequent message. Ideal solution It's clear that the ETH address should determine which fid should be allowed to claim it, not the fids. But for two verifications with the same claim, we can fall back on normal LWW mechanics.

Farcaster users keep their fid in an Ethereum address known as the custody address. The custody address (ECDSA keypair) issues a signed message which authorizes a signer (EdDSA keypair). The signer, in turn, can issue signed messages like casts or likes for the user. flowchart LR Root((ECDSA Keypair <br/> Custody Address <br/> 0x123)):::ecdsaSigner --> SignerAdd(Message <br/> authorize 0xabc <br/> signed_by: 0x123) SignerAdd:::message --> Signer((EdDSA Keypair <br/> Farcaster Signer <br/> 0xabc)):::eddsaSigner Signer --> Cast1(Message <br/> hello world <br> signed_by: 0xabc):::message Signer --> Cast2(Message <br/> it's me! <br> signed_by: 0xabc):::message classDef ecdsaSigner fill:#C9F0FF, stroke:#333; classDef eddsaSigner fill:#EFEFF0, stroke:#333;