Defra Foundations 101 [Archived]

--- tags: defradb, foundations, archived --- # Defra Foundations 101 [Archived] ## Defra Tech Docs - [DefraDB Query Specification](https://hackmd.io/@source/defradb-query) ## IPFS - [Proto School](https://proto.school/) - interactive tutorials on decentralized web protocols. ### IPLD - [IPLD](https://ipld.io/) - stands for InterPlanetary Linked Data. It is an ecosystem of formats and data structures for building applications that can be fully decentralized. ### MerkleDAGs - [Merkle-CRDTs: Merkle-DAGs meet CRDTs](https://arxiv.org/abs/2004.00107) ### (LibP2P) P2P ## CRDTs - [Conflict-free replicated data type](https://en.wikipedia.org/wiki/Conflict-free_replicated_data_type) - [YCombinator Discussion Forum](https://news.ycombinator.com/item?id=21464189) - [State of a State-based CRDT (Conflict-free Replicated Data Types)](https://bartoszsypytkowski.com/the-state-of-a-state-based-crdts/) - [Optimising State Based CRDTs - Part 1](https://bartoszsypytkowski.com/optimizing-state-based-crdts-1/) - [Optimising State Based CRDTs - Part 2](https://bartoszsypytkowski.com/optimizing-state-based-crdts-part-2/) - [Delta state-based CRDTs in Javascript - GitHub link](https://github.com/peer-base/js-delta-crdts) - [Dotted Version Vectors](https://riak.com/posts/technical/vector-clocks-revisited-part-2-dotted-version-vectors/) - [CRDT Notes](https://github.com/pfrazee/crdt_notes) - [CRDTs are the Future](https://josephg.com/blog/crdts-are-the-future/) - https://github.com/ipfs/notes/issues/406 - [CRDT: Conflict-free Replicated Data Types](https://medium.com/@amberovsky/crdt-conflict-free-replicated-data-types-b4bfc8459d26) - [Delta State Replicated Data Types](https://arxiv.org/abs/1603.01529) - [Merkle-CRDTs: Merkle-DAGs meet CRDTs](https://arxiv.org/abs/2004.00107) - [Consistency without Consensus: CRDTs in Production at SoundCloud](https://www.infoq.com/presentations/crdt-soundcloud/) - [A Bluffers Guide to CRDTs in Riak](https://gist.github.com/russelldb/f92f44bdfb619e089a4d) - [A comprehensive study of Convergent and Commutative Replicated Data Types](https://hal.inria.fr/file/index/docid/555588/filename/techreport.pdf) - [Open source collaborative text editors](https://juretriglav.si/open-source-collaborative-text-editors/) - https://news.ycombinator.com/item?id=19845776 - [Replicated Data Types](http://replicated.cc/rdts/) - [Implementation of Convergent and Commutative Replicated Data Types in Go.](https://github.com/neurodrone/crdt) ### Merkle CRDTs - [Merkle-CRDTs: Merkle-DAGs meet CRDTs](https://arxiv.org/abs/2004.00107) ## Datastore - github.com/ipfs/go-datastore ### General Database Internals - [Life of a SQL Query](https://sourcegraph.com/github.com/cockroachdb/cockroach@master/-/blob/docs/tech-notes/life_of_a_query.md) - [A Busy Developer’s Guide to Database Storage Engines — The Basics](https://blog.yugabyte.com/a-busy-developers-guide-to-database-storage-engines-the-basics/) - [A Busy Developer’s Guide to Database Storage Engines — Advanced Topics](https://blog.yugabyte.com/a-busy-developers-guide-to-database-storage-engines-advanced-topics) - [How We Built a High Performance Document Store on RocksDB?](https://blog.yugabyte.com/how-we-built-a-high-performance-document-store-on-rocksdb/) - [Building a large-scale distributed storage system based on Raft ](https://www.cncf.io/blog/2019/11/04/building-a-large-scale-distributed-storage-system-based-on-raft/) ### BadgerDB - [Badger DB](https://github.com/dgraph-io/badger) - is an embeddable, persistent and fast key-value (KV) database written in pure Go. It is the underlying database for Dgraph, a fast, distributed graph database. It's meant to be a performant alternative to non-Go-based key-value stores like RocksDB. ### Storage Engines - [RocksDB](http://rocksdb.org/) - is an embeddable persistent key-value store for fast storage - [LevelDB](https://github.com/google/leveldb) - is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values. - [Pebble](https://github.com/cockroachdb/pebble) - is a LevelDB/RocksDB inspired key-value store focused on performance and internal usage by CockroachDB. - [NutsDB](https://github.com/xujiajun/nutsdb) - is a simple, fast, embeddable and persistent key/value store written in pure Go. ### Alternate DBs - [ApacheCouchDB](http://couchdb.apache.org/) - Apache CouchDB™ lets you access your data where you need it. The Couch Replication Protocol is implemented in a variety of projects and products that span every imaginable computing environment from globally distributed server-clusters, over mobile phones to web browsers. ## GraphQL - [GraphQL](https://graphql.org/) - is a query language for APIs and a runtime for fulfilling those queries with your existing data. - [GraphQL Specification](https://spec.graphql.org/June2018/) ## CBOR - [Concise Binary Object Representation (CBOR)](https://cbor.io/) - is a data format whose design goals include the possibility of extremely small code size, fairly small message size, and extensibility without the need for version negotiation ## Query ### Planner - [Volcano-An Extensible and Parallel Query Evaluation System](https://paperhub.s3.amazonaws.com/dace52a42c07f7f8348b08dc2b186061.pdf) ## JSON PATCH - [JSON Merge Patch](https://datatracker.ietf.org/doc/html/rfc7386) - specification defines the JSON merge patch format and processing rules. The merge patch format is primarily intended for use with the HTTP PATCH method as a means of describing a set of modifications to a target resource's content. - [JSON Patch](http://jsonpatch.com/) - is a format for describing changes to a JSON document. It can be used to avoid sending a whole document when only a part has changed. When used in combination with the HTTP PATCH method, it allows partial updates for HTTP APIs in a standards compliant way. - [JSON Pointer](https://datatracker.ietf.org/doc/html/rfc6901) - defines a string syntax for identifying a specific value within a JavaScript Object Notation (JSON) document. ## Distributed Systems Reading - [Partition Tolerance](https://codahale.com/you-cant-sacrifice-partition-tolerance/) - [Distributed Systems Reading List](https://dancres.github.io/Pages/) ## Misc - [Replicate Your Database to the Client](https://josephg.com/blog/replicate-your-database-to-the-client/) - [ The database I wish I had ](https://euandre.org/2020/08/31/the-database-i-wish-i-had.html) - https://news.ycombinator.com/item?id=24337244 - [Using Bloom filters to efficiently synchronise hash graphs](https://martin.kleppmann.com/2020/12/02/bloom-filter-hash-graph-sync.html)