A few statements that we can all agree are true:
- As per Logos, Codex is the "storage" aspect of the "holy trinity" w.r.t. decentralized infrastructure
- Codex's main differentiator in the DSN ecosystem is "durability"
- Durability _with strong guarantees_ has an associated cost
- The landscape of "storage" needs is greater than a need for strong durability
- "Durability" within Codex is provided by multiple parts coming together: P2P Erasure Coding (core of it all), Marketplace, Storage Proofs, Automatic Repair
- All of these projects have no value if there is no community
- the communication infrastructure for regularly setting expectations and continuously updating them has been lacking
- shipping a large complex project all at once is riskier than shipping it's individual pieces over time
- production engineering and launching a product is hard work in itself

## Codex as the storage pillar of Logos, a continuation of Ethereum's "Holy Trinity"

Let's start with what the "Holy Trinity" is, which begins with what infrastructure is consumed in applications and how that infrastructure changes in the context of "distributed systems" and the guarantees that are strived for when making that transition.

A given application will consume infrastructure in the following forms:
- computation
- communication
- storage

The traditional application industry has primarily focused around scaling the ability to scale on relied upon infrastructure by centralizing the ownership and trust. This has lead to a tremendous amount of power and influence to the infrastructure owners, which are few and their motivation is profit. This then leading to application development that optimize for extracting data and value from users. Users are the product.

That being said, this infrastructure development lead to traditional distributed systems. This development was an attempt to scale performance while adding an underlying resilience and consistency. As an example, a million clients hitting a database that exists on a single server will lead to failures, or if that single server fails for other reasons, all clients that rely on it will fail to perform their function.

This lead way to innovations like RAFT and PAXOS, and lead to differential terms like weak and strong consistency. While it enabled massive scale applications that we experience today, it also complicated the development process and associated guarantees of infrastructure that could be provided. An example of a distributed file storage system is [GFS](https://static.googleusercontent.com/media/research.google.com/en//archive/gfs-sosp2003.pdf).

The vision of Ethereum's "Holy Trinity" came from the attempt to distribute the power and ownership of this infrastructure while maintaining the relative performance and safety guarantees. It is a continuation of the ongoing generalization of distributed systems infrastructure. It was clear that previous attempts to scale remained to have control and influence of these massive systems in the hands of very few. They were resilient, but they weren't democratized commensurate with their value and power.

### Codex as the Storage pillar
Storage within the context of application infrastructure is effectively: I put something somewhere for an amount of time and have expectations that I can retrieve it when I need it within that timeframe.

The guarantees around the time it should be there and the likelihood of successful retrieval are the details where the devil lies.

Since inception, Codex has strived to differentiate itself from other Decentralized Storage Networks (DSN) by providing a utility none of them have successfully done: efficient durability. This comes at the cost of multiple complex systems working in tandem in order to provide high availability guarantees over the lifetime of a given storage contract in the network.

We can break down how the Codex Durability Guarantee is provided by looking at the individual systems that come together to make it up:
- p2p erasure coding and dispersal (core network function)
- contract marketplace and incentives
- remote auditing of storage availability
- automatic repair

Without all of these systems working properly, then the durability guarantees _CANNOT_ be provided, and thus the differentiation factor of the overall network is lessened. That being said, each of these pieces adds layers of additional complexity, both through the piece itself, and the interactions with the other pieces when put together.

That being said, high durability guarantees are but only a slice of the totality of storage needs within a decentralized application context. The Codex team has stated that the system _could_ cater to these needs, but it has yet to be seen how exactly this would be done, and how the current system would need to be altered to efficiently do so.

## Some Thoughts About Shipping Large Complex Projects
There is a significant risk when launching a large, complex project all in one go. Put simply, there will be bugs. The larger and more complex the project is, the more difficult it is to identify the cause and associated effects of a given bug. This is greatly exacerbated in distributed networks as simple secure-SDLC procedures don't cover network and independent actor related issues.

A less risky approach is to productionize and market test individual pieces as they're completed and hardened, while continuing to research and develop the remaining pieces in parallel.

Assuming that each piece only provides a fraction of the total guarantees/features of the total picture, thus only enabling a subset of the total product functionality.

Below is a diagram looking at "the ephemerality of a given peice of data" and how our current Logos Products map to it:


Because the differentiator of Codex is a focus on efficient durability, it is assumed that the full delivery that is planned primarily focuses on the end of this spectrum. The earlier phases (e.g. non-incentivized testnet) is a release that focuses on the middle to middle-right of the diagram as it doesn't give as much of a guarantee of the availability of the data.

A consequence of not having as high of durability guarantees in this less functional product is that the standards of the users that consume it are lessoned as well. It doesn't promise the world, so the consumer doesn't expect it, giving us an opportunity to meet and exceed the demands of a more relaxed userbase. This allows us to be sure that the parts of the system that are more straightfoward are hardeneed and stable without heavy user demands and expectations.

Furthermore, Codex is a complicated project. The more complicated something is, the more difficult it is to reason about and keep a clear picture in your head. By hardening off core components and ensuring they work as intended, you free up your brain to keep the parts still under research and development more clear and manageable.

There is also an aspect of growing a community by delivering products that they can get their hands on and do something with. This is a skill we have yet to develop well as an organization, so we should be trying to do it _as much as possible, with every product we trying to ship._
### A Proposed Piecemeal Codex Deployment
The following is a proposed development and delivery roadmap that is an alternative to the current Codex roadmap that focuses on hardening off parts of the codebase and creating products and a community of users around them. Each subsequent product builds on the previous one, adding complexity and features along the way (Thank you Jacek for the contributions here).

1. work towards altruistic codex for use cases where the data has intrinsic value (and therefore doesn't need incentives)
- The suggested path forward here is basically non-systemic erasure coding which has redundancy and protects node operators since it no longer can be reconstituted
- present the network as "ready" at this point, ie start building use cases, launch as v1, publish the rest as a detailed roadmap.
- the fit here was status communities archival or something like this, but also historical data of any sort (wikipedia, anna's archive etc)
- we could sell this to l2's and block chains in general already and it would mesh with the ETH data availability research using the right buzzwords and all
2. build out repair - this implies gossiping block availability and replaces the proof scheme with a gossip mechanism
- developing it this way ensures that the "volume" of "I have the data" is sane
- It is clear the current plan is via ZK-based proof aggregation
- Any advanced scheme based on ZK and so on is entirely replaceable with a "classic" scheme, ie instead of developing a proof for "i have data" the node can simply bls-sign "I have data"
- or even not sign and just gossip "I have data" - if the volume of traffic here is "too much", it will also be too much on a blockchain, posting proofs
3. work towards ensuring node operator plausible deniability etc - ie if we need to add filtering, clearly separate it from the base layer, stick it in some gateway or something (so that cloudflare can operate a public gateway but the underlying network is agnostic)
- the non-systemic code should get us 80% there but there might be more to do (ie classic "only-gateway-knows-the-decryption-key" schemes)
- maybe we can leave it at that and return to it later with a more powerful scheme (some seed rotation or something like that would be cool, if it could be combined with PFS - not sure if this is doable)
4. (re)develop the marketplace
- last year, the marketplace was insanely inefficient because it stored ephemeral stuff ("I have data") on chain
- instead, it would do well to use the above developed gossip mechanism for any "affirmitive" stuff and focus on conflicts which would bring down costs
- or ensure that the underlying storage for "I have data" is off-chain (stored itself on codex?) while the rest goes into data blobs or a similar temporally limited cheap construct
- focus on challenges instead, like optimistic rollups
- recombine the above with the repair mechanism etc that users can opt in to incentives, maybe launch this as v2

## The Current State of Affairs
Currently, the Codex team is working on multiple fronts:
- stabilizing the client for public testnets. This is the lion's share of the work being done. The current testnets being prepped for deployment are:
- non-incentivized: the "p2p network" which is ostensibly the "altruistic network" which doesn't included the marketplace, storage proofs, or repair.
- incentivized: inclusion of the mvp versions of the whole system, with known improvement paths for each component.
- working out the on-chain proof verification cost
- currently, costs of the proof verification via smart contracts is too high for efficient economics
- looking into sidechains, precompiles, etc
- this work is mostly orthogonal to other on-going work
- this has been estimated to be "understood" by September. If not, then re-focus on prioritizing the "altruistic network" would be appropriate.
- The marketplace is being continued by Eric and Marcin, peripherally by Mark
- Mark is researching blockchain deployment options and not spending much time on the Marketplace development
- Marcin is a new hire, and is in the process of ramping up before he becomes effective.
- Self-repair - N/A (afaik)

The main question at hand is what to prioritize, and when key decisions need to be made to switch:
1. productionizing the "altruistic network" and rolling that out as the initial Codex product (as per the above or similar release plan)
2. completing the entire package and shipping that as currently intended

It has been said that the current state of development is so close to being completed that it makes more sense to do option 2 now, whereas option 1 would be been more appropriate a year or two ago.